Relative entropy via non-sequential recursive pair substitution

Relative entropy via non-sequential recursive pair substitutions

The entropy of an ergodic source is the limit of properly rescaled 1block entropies of sources obtained applying successive non-sequential recursive pairs substitutions [7],[2]. In this paper we prove that the cross entropy and the KullbackLeibler divergence can be obtained in a similar way. AMS classification scheme numbers: 94A17

Non-sequential Recursive Pair Substitutions and Numerical Entropy Estimates in Symbolic Dynamical Systems

We numerically test the method of non-sequential recursive pair substitutions to estimate the entropy of an ergodic source. We compare its performance with other classical methods to estimate the entropy (empirical frequencies, return times, Lyapunov exponent). We considered as a benchmark for the methods several systems with different statistical properties: renewal processes, dynamical system...

Nonsubjective Priors via Predictive Relative Entropy

We explore the construction of nonsubjective prior distributions in Bayesian statistics via a posterior predictive relative entropy regret criterion. We carry out a minimax analysis based on a derived asymptotic predictive loss function and show that this approach to prior construction has a number of attractive features. The approach here differs from previous work that uses either prior or po...

Nonsubjective Priors via Predictive Relative Entropy Regret

We explore the construction of nonsubjective prior distributions in Bayesian statistics via a posterior predictive relative entropy regret criterion. We carry out a minimax analysis based on a derived asymptotic predictive loss function and show that this approach to prior construction has a number of attractive features. The approach here differs from previous work that uses either prior or po...

Iterated Relative Recursive Enummmerability